1. Field of the Invention
The present invention relates to an image processing apparatus and method, particularly to an apparatus for and a method of extracting (cutting out) a desired image portion from an original image displayed on a screen of a display device, and an apparatus for and a method of further extracting a part of the partial image, moving the part of the image, and arranging (editing) or storing a plurality of partial images upon correlating with each other.
Further, the present invention relates to an image synthesizing system comprising a client computer and an image synthesizer which can communicate data to each other and an image synthesizing method in the image synthesizing system, the client computer and the image synthesizer which constitute the image synthesizing system, and an image separating method.
2. Description of the Background Art
The format of image data includes GIF (Graphic Interchange Format), EPSF (Encapsulated PostScript Format), and so forth.
A color is expressed using a color palette (e.g. of 256 colors) in the GIF. A desired part of an image (a partial image) can be specified in the original image of the GIF format. The contour of the partial image can be designated as desired. An area (background) other than the specified partial image in the original image is represented by a set of image data of a transparent color. The GIF format has such inconvenience that the specified partial image and the background image are always treated as an integrated image data. Further, an image which is expressed using colors close to colors in nature (e.g., 16,000,000 colors) (referred to as a natural image or a picture of a scene) cannot be represented by the GIF format.
The EPSF is suited for graphics and graphics is expressed by a vector. Since a picture of a scene hardly be expressed by a vector, the picture of a scene cannot be also handled in the EPSF format.
On the other hand, when a subject image (a user image) picked up by a still video camera, a movie video camera or the like is fetched into a computer, and a color image is printed using a color printer, image data representing the user image is subjected to color correction depending on the printing characteristics of the color printer.
With the development of the computer, it has been possible for a user himself or herself to inlay the user image, in a position, in which a user image is to be inlaid, on a template image representing the background of the user image. A composite image obtained by inlaying the user image in the template image can be also printed using a color printer that the user has.
Even in such a case, the characteristics of the color printer is known by the user, and the user himself or herself stores in the computer image data representing the user image and image data representing the template image. Therefore, it is possible to separately take out the user image data and the template image data, and separately subject the user image data and the template image data to color correction depending on the characteristics of the color printer prior to the synthesis of the user image and the template image.
The user image data which has been subjected to the color correction and the template image data which has been subjected to the color conversion are synthesized to produce the composite image data, so that the composite image obtained by the printing exhibits proper colors as a whole.
Consider an image synthesizing system comprising a client computer and an image synthesizer located at a place spaced apart from the client computer. In such a system, when a template image and a user image are synthesized as described above in the client computer, image data representing a composite image is transmitted from the client computer to the image synthesizer, and the composite image is printed using a color printer connected to the image synthesizer, it is difficult to separate the template image and the user image from the composite image in the image synthesizer. Since it is difficult to separate the template image and the user image from the composite image it is difficult to individually perform the most suitable color conversion (inclusive of color correction) conforming to the template image and the most suitable color conversion (inclusive of color correction) conforming to the user image in the image synthesizer.
When the whole of the composite image is subjected to color conversion, even if the color conversion is most suitable for the template image, the color conversion may be, in some cases, unfavorable for the user image, or vice versa.
An object of the present invention is to make it possible to handle, when a desired partial image having a desired shape is extracted (cut out) from an image, only the extracted partial image, i.e., independently from a background image (the remaining image portion).
Another object of the present invention is to make it possible to designate and extract (cut out) a partial image from not only a natural image (a picture of a scene) but also an image represented using a color palette.
Still another object of the present invention is to make it possible to further extract a part of the extracted partial image to obtain a plurality of partial images, and to define the positional relationship among the plurality of the partial images obtained by the extraction.
Still further object of the present invention is to make it possible to perform, even after a template image has been inlaid in a user image to produce a composite image, color conversion suitable for the template image and the user image individually.
The first invention is directed to an image processing apparatus, which comprises an image area designating device for designating an area to be extracted of an original image displayed on a display screen, color changing means for changing the color of an area, excluding the area to be extracted which has been designated by the image area designating device, of the displayed original image into a particular color, and storage controlling means for storing pixel data representing pixels having colors excluding the particular color obtained by the change in the color changing means in the displayed original image and coordinate data representing positions of the pixels in the displayed original image with the pixel data and the coordinate data correlated with each other.
The first invention provides an image processing method which comprises the steps of designating an area to be extracted of an original image displayed on a display screen, changing the color of an area, excluding the designated area to be extracted, of the displayed original image into a particular color, and storing pixel data representing pixels having colors excluding the particular color obtained by the change in the displayed original image and coordinate data representing positions of the pixels in the displayed original image with the pixel data and the coordinate data correlated with each other.
The first invention further provides a storage medium storing thereon a program for controlling a computer, the program causing the computer to accept an area to be extracted which is designated on an original image displayed on a display screen, to replace image data representing an image of an area excluding the accepted area to be extracted with image data representing a specified color on the image data of the original image, to generate pairs of pixel data and coordinate data from the image data excluding the image data representing the specified color, and to store the generated pairs of the pixel data and the coordinate data on a storage medium.
According to the first invention, since the image data in the area (background area) other than the area which has been specified to be extracted (cut out) are replaced with image data representing the particular color, the image data of the specified (designated) area can be easily obtained merely by extracting other image data than the image data which represent the particular color.
The pixel data (pixel image data) and the coordinate data are produced based on the extracted image data and are stored in correlation with each other. The extracted image data have coordinate data representing the position of the pixels in the image for each pixel, so that the extracted image data alone, that is, without accompanying image data representing the background, can be treated or handled. The image data representing the background can be dispense with or unnecessary. The coordinate data is for specifying the relative positions of the pixels in image. The coordinate data may be defined using any point on the extracted image (inclusive an area including the extracted image, e.g. a rectangle which circumscribes the extracted image) as an origin of the coordinate.
According to the first invention, it is possible to designate and to extract a partial image from not only an image represented using a color palette but also a natural image.
In the preferred embodiment, the pixel data and the coordinate data form pairs to be stored.
In another embodiment, a set of the pixel data is compressed, and a set of the coordinate data is compressed. The compressed pixel data and the compressed coordinate data are correlated with each other to be stored. The pixel data and the coordinate data are individually subjected to data compression respectively conforming to the characteristics or nature of the pixel data and coordinate data, so that efficient data compression can be achieved.
The first invention further provides a recording or storage medium having the above image data (pixel data and coordinate data) stored thereon.
One of the image data storage medium according to the first invention is the storage medium storing thereon pairs of pixel data and coordinate data representing pixels composing an image in the order represented by the coordinate data.
Another of the image data storage medium according to the first invention is the storage medium storing thereon compressed pixel data and compressed coordinate data which are obtained by respectively compressing such a group of pixel data and a group of coordinate data that the pixel data and the coordinate data representing pixels composing an image are arranged for pixel data and for coordinate data in the order represented by the coordinate data.
An apparatus for reproducing an image represented by the image data which has been stored in accordance with the first invention comprises data reading means for reading, from a storage medium storing image data comprising a set of pairs of pixel data representing pixels and coordinate data representing the positions of the pixels on the image, the pixel data and the coordinate data, and display controlling means for controlling a display device such that the image is displayed by displaying the pixels represented by the pixel data read by the data reading means in the positions represented by the coordinate data.
A method of reproducing an image comprises the steps of reading, from a storage medium storing image data comprising a set of pairs of pixel data representing pixels and coordinate data representing the positions of the pixels on the image, the pixel data and the coordinate data, and controlling a display device such that the image is displayed by displaying the pixels represented by the read pixel data in the positions represented by the corresponding read coordinate data.
An apparatus for reproducing an image represented by the compressed image data which has been prepared and stored in accordance with the first invention comprises data reading means for reading, from a storage medium storing thereon compressed pixel data and compressed coordinate data which are obtained by compressing a set of pixel data and by compressing a set of coordinate data in image data having a data structure such that pixels constituting an image are represented by pixel data and coordinate data, the compressed pixel data and the compressed coordinate data, pixel data expanding means for expanding the compressed pixel data read by the data reading means, coordinate data expanding means for expanding the compressed coordinate data read by the data reading means, and display control means for controlling a display device such that the image is displayed by displaying the pixels represented by the pixel data expanded by the pixel data expanding means in the positions represented by the coordinate data expanded by the coordinate data expanding means.
A method of reproducing the compressed image data comprises the steps of reading, from a storage medium storing thereon compressed pixel data and compressed coordinate data which are obtained by compressing a set of pixel data and by compressing a set of coordinate data in image data having a data structure such that pixels constituting an image are represented by pixel data and coordinate data, the compressed pixel data and the compressed coordinate data, expanding the read compressed pixel data, expanding the read compressed coordinate data, and controlling a display device such that the image is displayed by displaying the pixels represented by the expanded pixel data in the positions represented by the expanded coordinate data.
The second invention is directed an image processing apparatus, which comprises a designating device for designating on an image displayed on a display screen based on image data comprising pixel data representing pixels and coordinate data representing the positions of the pixels, a partial image to be moved, a movement value inputting device for inputting the direction of movement and the amount of movement of the partial image, movement amount adding means for adding the coordinate variation corresponding to the amount of movement in the direction of movement inputted by the movement value inputting device to the coordinate data of the pixels composing the partial image designated by the designating device, and storage controlling means for storing the new coordinate data obtained by the movement amount adding means and the pixel data corresponding thereto in correlation with each other as well as the coordinate data and the pixel data representing the image other than the partial image.
An image processing method according to the second invention comprises the steps of designating on an image displayed on a display screen based on image data comprising pixel data representing pixels and coordinate data representing the positions of the pixels, a partial image to be moved, inputting the direction of movement and the amount of movement of the partial image, adding the coordinate variation corresponding to the inputted amount of movement in the inputted direction of movement to the coordinate data of the pixels composing the designated partial image, and storing the new coordinate data obtained by the addition and the pixel data corresponding thereto in correlation with each other as well as the coordinate data and the pixel data representing the image other than the partial image.
The second invention further provides a storage medium storing thereon a program for controlling a computer, the program causing the computer to accept an area of a partial image designated on an image displayed on a display screen based on image data comprising pixel data representing pixels and coordinate data representing the positions of the pixels, to accept the direction of movement and the amount of movement of the partial image, to calculate new coordinate data by adding the coordinate variation corresponding to the accepted amount of movement in the accepted direction of movement to the coordinate data of the pixels composing the partial image of the accepted area, and to store the new coordinate data obtained by the calculation and the pixel data corresponding thereto in correlation with each other as well as the coordinate data and the pixel data representing the image other than the partial image.
In a preferred embodiment of the second invention, the partial image before movement disappears, and the partial image after movement appears at the positions defined by the new coordinate data on the screen of the display device. The user can recognize the position of the moved partial image.
According to the second invention, even in a case where a part of an image is extracted (cut out) and the extracted partial image is moved, the pixel data and the new coordinate data which represent the moved partial image are correlated with each other to be stored together with the pixel data and the coordinate data of the original image (exclusive of the moved partial image).
Preferably, all of the above pixel data and the coordinate data are stored in the predetermined order, e.g., the order represented by the coordinate data. A set of the pixel data and a set of the coordinate data are individually compressed, and the compressed pixel data and compressed coordinate data are combined to be stored.
In this way, in a case where a part of an image is extracted and is moved, the image data representing these images (the extracted partial image and the original image exclusive of the extracted partial image) are saved in a state that the positional relationship of these images is kept. It is possible to handle or treat a plurality of images which are separated from each other with maintaining the mutual positional relationship therebetween.
An image synthesizing system according to the third invention comprises a client computer and an image synthesizer which can communicate data to each other.
The client computer comprises an image synthesizing device for inlaying a user image or a portion thereof, in a position defined by mask information accompanying a template image, on the template image representing the background of the user image, and a composite image information transmitting device for transmitting composite image data representing a composite image produced by the image synthesizing device and the mask information used for the synthesis, upon correlating with each other, to the image synthesizer.
The image synthesizer comprises a composite image information receiving device for receiving the composite image data and the mask information which have been transmitted from the client computer, and an image data separating device for separating at least one of image data representing the template image and image data representing the user image from the composite image data representing the composite image, on the basis of the received mask information.
The third invention also provides a method suitable for the image synthesizing system. That is, the method is an image synthesizing method in the image synthesizing system comprising the client computer and the image synthesizer which can communicate data to each other.
In the client computer, a user image or a part thereof is inlaid in a position defined by mask information accompanying a template image, on the template image representing the background of the user image, and the composite image data representing a composite image obtained by inlaying the user image in the template image and the mask information used for inlaying are correlated with each other and are transmitted from the client computer to the image synthesizer.
In the image synthesizer, the composite image data and the mask information which are transmitted from the client computer are received, and at least one of image data representing the template image and image data representing the user image, which constitute the composite image data representing the composite image are separated, from the composite image data on the basis of the received mask information.
According to the third invention, the user image is inlaid in the position defined by the mask information representing the position, in which the user image is to be inlaid, on the template image to produce the composite image in the client computer. The composite image data representing the composite image and the corresponding mask information are correlated with each other, and are transmitted from the client computer to the image synthesizer.
In the image synthesizer, the composite image data and the mask information, which have been transmitted from the client computer, are received. The position, in which the user image is to be inlaid, on the template image is recognized on the basis of the received mask information. Since the position in which the user image is inlaid is found, the template image data and the user image data can be relatively easily separated from the composite image data.
Since the template image data and the user image data, which constitute the composite image data, are separately obtained, the template image data and the user image data can be separately and individually subjected to color conversion, as required. It is possible to perform the most suitable color conversion for the template image data and the most suitable color conversion for the user image.
The composite image is produced again from the template image data and the user image data at least one of which has been subjected to the color conversion, as required. A re-composite image thus obtained by the resynthesis is printed in a printer. Since at least one of the template image and the user image, which constitute the re-composite image, is individually subjected to the color conversion, the re-composite image whose colors have been converted to proper colors is printed in the printer.
The foregoing and other object, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.